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ABSTRACT 



Multimedia learning environments commonly comprise different 
sources of information such as various linguistic as well as static and 
dynamic pictorial representations. In this context, it is frequently assumed 
that the (simultaneous) presentation of different and possibly complementary 
sources of information improves learning. During the last 10 years, however, 
psychological and educational research have collected extensive evidence that 
the presentation of different sources of information in multimedia learning 
environments might not improve and even impede learning. If multimedia 
learning is to be successful, different sources of information need to be 
integrated into coherent mental representations. In many cases, however, the 
presentation and use of different sources of information does not result in 
the construction of appropriate mental representations. Starting from various 
potentials and problems of multimedia learning, the theory of structure 
mapping is proposed as a framework that can be used to design multimedia 
learning environments that systematically encourage and support the mental 
integration of multiple sources of information. A multimedia learning 
environment for statistics is described which has been implemented according 
to the design principles proposed. (Contains 10 references.) (AEF) 
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Abstract: If multimedia learning is to be successfull, different sources of information need to be 
integrated into coherent mental representations. In many cases, however, the presentation and use 
of different sources of information does not result in the construction of appropriate mental 
representations. Starting from various potentials and problems of multimedia learning, the theory of 
structure-mapping is proposed as a framework which can be taken advantage of to design 
multimedia learning environments which systematically encourage and support the mental 
integration of multiple sources of information. A multimedia learning environment for statistics is 
described which has been implemented according to the design principles proposed. 



Introduction 

Multimedia learning environments commonly comprise different sources of information such as various linguistic as 
well as static and dynamic pictorial representations. In this context, it is frequently assumed that the (simultaneous) 
presentation of different and possibly complementary sources of information improves learning. During the last 10 
years, however, psychological and educational research have collected extensive evidence that the presentation of 
different sources of information in multimedia learning environments might not only not improve but even impede 
learning. In many cases, the „obvious“ benefits of multimedia learning environments vanish in the moment serious 
evaluation takes place. 

In current psychological and educational theories it is hypothesized that different sources of information are 
processed by different mental systems (e.g,, Mayer, 1997; Paivio, 1986; Schnotz & Bannert, 1999). It is proposed 
that linguistic information is processed by a verbal system and pictorial information is processed by a visual system. 
The theories differ in their assumptions about the connections between the verbal and visual system. 

According to Paivio’s (1986) dual coding theory, the construction of mental representations of linguistic information 
only takes place in the verbal system. The construction of mental representations of pictorial information, in 
contrast, takes place in the visual as well as in the verbal system. Mayer (1997) assumes that mental representations 
constructed by the verbal system are mapped to mental representations constructed by the visual system as well as 
the other way round, Schnotz and Bannert (1999) propose that mental representations of linguistic and pictorial 
information are integrated into a single mental model which corresponds to an analogous and modality-unspecific 
mental structure. 
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According to the theories of Mayer (1997) and Schnotz and Bannert (1999), successful learning in multimedia 
learning environments aims at the integration of linguistic and pictorial information into coherent mental 
representations. However, the presentation and use of different sources of information frequently does not result in 
the construction of appropriate mental representations. Up until now, it has remained an open question how the 
construction of coherent mental representations can adequately be supported in multimedia learning. 

In the following, various potentials and problems of learning with different sources of information are summarized. 
Afterwards, the theory of structure-mapping is proposed as a framework which can be taken advantage of to design 
multimedia learning environments which encourage and support the mental integration of multiple sources of 
information and to construct coherent mental representations. Finally, the multimedia learning environment 
VISUALSTAT for statistics is described. This learning environment has been implemented according to the design 
principles proposed. 



Potentials and Problems of Learning with Multiple Sources of Information 

Potentially, multiple sources of information may improve learning in multimedia learning environments in various 
ways. For instance, different sources of information may be complementary, resulting in the construction of more 
complete mental representations of an application domain than a single source of information (e.g., Ainsworth, 
1999). Furthermore, multiple sources of information may realize different conceptual perspectives on an application 
domain leading to the construction of multiple mental representations which can flexibly be used during problem 
solving, for example (e.g., Spiro & Jehng, 1990). 

During the last 10 years, however, psychological as well as educational research demonstrated that the presentation 
of different sources of information in multimedia learning environments poses various problems to the students. 
Each source of information may rely on notations which are not familiar to the students. For instance, in many cases 
students are not able to identify visual and spatial structures in pictorial representations which would allow them to 
understand an application domain (e.g., Lowe, 1998). In other cases, students do not interpret the perceived visual 
and spatial structures conceptually (e.g., Weidenmann, 1994). Very often, such a lack of visual literacy is 
accompanied with illusions of understanding (e.g., Salomon, 1994). 

The need to process different sources of information poses additional challenges to the students. For example, the 
students have to process larger amounts of information and to direct their attention simultaneously to different 
information (e.g., Lowe, 1998). Very often, these requirements overburden the students’ cognitive capabilities 
resulting in only little learning (e.g., Sweller, 1993, 1994). 

One of the most severe problems, however, may be the finding that students frequently do not systematically relate 
different sources of information to each other (e.g., Ainsworth, Bibby & Wood, 1996; Anzai, 1991; Kozma et al., 
1996; Lowe, 1998; Peeck, 1993). As a consequence, these students fail to integrate the information into coherent 
mental representations. Their mental representations about an application domain remain fragmentary and - 
especially - disjointed. During problem solving, for instance, these students very often switch back and forth 
between different mental representations of a posed problem without being able to determine which representation 
contributes in which ways to the problem’s solution (e.g., Anzai, 1991). 

Although the use of multiple sources of information has the potential to deepen understanding, their coordination 
and integration does not take place on its own. 



Facilitating the Integration of Multiple Sources of Information by Structure Mapping 

We propose the theory of structure-mapping as a framework which can be taken advantage of to design multimedia 
learning environments which encourage and support the mental integration of different sources of information. 
Initially, Gentner (1983) modeled the construction of analogies between two application domains by means of 
structure-mapping. A structure is made up of objects and their attributes as well as relations between objects. 
According to Gentner’s (1983) model, an analogy is constructed by (partially) mapping the mental structure which 
represents the familiar domain, named the base, onto the mental structure which represents the unfamiliar domain, 
named the target. In recent years, Gentner and Markman (1997) demonstrated how structure -mapping can be used to 
model how similarities among different objects are established. 

We apply the model of structure-mapping as a means in order to encourage and support students to (1) externally 
relate different sources of information to each other and - as a consequence - to (2) mentally integrate the different 
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sources of information. In many cases, different sources of information in multimedia learning environments 
delineate structures which are (partially) related to each other. Because students rarely identify relevant structures 
and establish relations between them spontaneously, the model of structure-mapping can serve as a framework in 
order to guide the implementation of mechanisms into a multimedia learning environment which interactively and 
systematically (1) encourage the identification of structures relevant to an application domain and (2) support the 
construction of relations between the identified structures by (partially) mapping one structure onto the other. 

In earlier research it has been demonstrated that students who actively relate different sources of information to each 
other construct more complete and more coherent mental representations of the application domain under scrutiny 
than students who do not (e.g., White, 1993). To know the relations between different sources of information also 
allows one to flexibly switch between them on the one hand and to integratively make use of them on the other hand 
during problem solving, for example (e.g., Anzai, 1991). Structure-mapping forms one means to guide the 
construction of such relations. 
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Figure 1. Different sources of information about the model and the methods underlying the one-way analysis of 
variance with two groups. 



Supporting the Integration of Multiple Sources of Information in VISUALSTAT 

VISUALSTAT is a multimedia learning environment for statistics for the social sciences in the World Wide Web 
(www.psychologie.uni-freiburg.de/visualstat/). It comprises different sources of information about various models 
and methods of statistics such as the general linear model, the method of least squares and the partition of the sum of 
squares. These models and methods underlie frequently applied procedures in statistics such as the t-test, the (one- 
way) analysis of variance and the (simple) linear regression (e.g.. Hays, 1994). VISUALSTAT* does not make up a 
complete and self-contained learning environment which is intended to be used instead of a textbook. It rather 
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complements a textbook. While textbooks describe the considered models and methods almost always exclusively 
by means of text and algebraic equations, VISUALSTAT additionally visualizes them dynamically and interactively 
(see Figure 1). The dynamic and interactive visualizations draw a student’s attention to properties of the considered 
models and methods which frequently remain unrecognized or misunderstood in traditional statistics courses. 
VISUALSTAT can be used in two different modes: (1) self-guided and (2) guided learning. The mode of self- 
guided learning aims at students who are already familiar with the dynamic and interactive visualizations in 
VISUALSTAT. These students are encouraged to take advantage of the different sources of information according 
to their own learning objectives and information needs. 

The mode of guided learning aims at students who are unfamiliar with the dynamic and interactive visualizations. 
These students receive guidance according to the design principles described above. Firstly, in order to avoid 
cognitive overload, one visualization after the other is presented to the students. 

Secondly, the students are encouraged to interactively and systematically map step by step components of the 
(familiar) textual and algebraic representations to components of the (unfamiliar) graphical representations in order 
to learn (see Figure 2): (1) which components make up the visualizations, (2) how the text, the algebraic equations 
and the visualizations are related to each other as well as (3) how the different visualizations are related to each 
other. 

Every time a mapping is completed, the students are informed about the correctness of their mapping. If students 
repeatedly construct an incorrect mapping, the correct one is explained to them. 
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Figure 2. Interactive mapping of components of textual and algebraic representations to components of graphical 
representations underlying the mean. 



Discussion 

Although multimedia learning environments potentially may improve students’ learning in various ways, they also 
may impede learning. For instance, students frequently do not systematically relate different sources of information 
to each other in multimedia learning. As a consequence, their mental representations about an application domain 
remain fragmentary and disjointed. 
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In empirical studies which revealed these problems it is very often concluded that learning in multimedia learning 
environments needs to be supported (e.g., Ainsworth, Wood & Bibby, 1996; Peeck, 1993). However, until now, only 
a few measures have been proposed to support multimedia learning (e.g., Mayer, 1997; Sweller, 1993, 1994). The 
measures which have been proposed keep the students in perceptive and rather passive roles. They focus on how 
different sources of information should be presented to the students in oder to facilitate the students’ learning (e.g., 
Mayer, 1997; Sweller, 1993, 1994). 

In this paper, we proposed a form of support which places the students in active roles. It focuses on how the 
students’ mental processing of the presented information can systematically be encouraged and supported. Our 
proposal was based on the theory of structure-mapping (cf. Gentner, 1993). According this theory, two sources of 
information can be related to each other by (partially) mapping structures from one source of information onto 
structures in another source of information. 

The multimedia environment VISUALSTAT was equipped with mechanisms which implement the proposed 
principles. In VISUALSTAT, the students are guided to relate different sources of information to each other by (1) 
identifying the relevant structures in the information presented and (2) constructing mappings between the identified 
structures. We are convinced that the activity of systematically relating different sources of information to each 
other will result in the construction of more complete and more coherent mental representations. Currently, we are 
designing an experimental study which will be conducted in order to formally evaluate the learning effects of the 
support mechanisms implemented in VISUALSTAT. 
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